Digital cameras consist of hardware components necessary to capture an image along with computational elements required to convert the captured signals to a digital record of the scene. In order to prepare the camera system for capture, all digital cameras have some means for determining exposure settings responsive to scene light level. Conventional solutions for determining exposure settings typically use one or more standardized settings, or respond to operator mode settings to obtain an exposure setting. FIG. 1 shows a flow chart of a typical exposure control system 200 for a digital camera performing autoexposure. In assess scene brightness step 210, the camera assesses the scene brightness either with a scene brightness sensor or with an analysis of a preview image. In determine capture mode step 220, a capture mode setting 225 is determined based on the measured scene brightness and any operator-selected user interface settings or standardized settings. In determine exposure index step 230, the exposure index setting 235 (EI) is determined in accordance with the measured scene brightness and the capture mode setting 225. Those skilled in the art will recognize that exposure index is a standard way to quantify the amount of light necessary for a good exposure. For film-based cameras, the exposure index is usually set based on the film speed, or ISO rating, which is related to the film sensitometry. Conversely, in digital cameras, the exposure index (EI) is often set based on a number of factors including scene brightness, and the effective ISO of the digital camera is adjusted to largely match the EI. In determine aperture step 240, an aperture setting 245 is determined to control the F/# of the camera lens in accordance with the measured scene brightness, the capture mode setting 225 and the exposure index setting 235. An exposure time setting 255 (t) is then determined in determine exposure time step 250 in accordance with the scene brightness, the capture mode setting 225, the exposure index setting 235 and the aperture setting 245. The capture mode setting 225, the exposure index setting 235, the aperture setting 245 and the exposure time setting 255 can be collectively referred to as image capture settings 270.
It should be noted that these steps are not necessarily performed in the order shown in FIG. 1. After the image capture settings 270 have been determined, a capture digital image step 260 is used to capture and store a digital image 265. However, the method of the typical camera control system 200 is prone to capture images with poor perceived image quality because the degree of brightness and motion in the scene can be highly variable and since motion is not taken into account, disappointing levels of motion blur or noise can be present in the images.
Various methods have been employed to determine the optimal combination of the exposure settings responsive to a particular scene. In the simplest implementation, specific values for F/#, EI and t are specified for each light level to produce the Exposure Program Curve for the camera system. More sophisticated implementations may include varying the exposure settings as a function of other factors such as scene type, zoom position and capture mode. For example, it is known that blurring effects due to camera motion (e.g., motion caused by camera shake) is amplified by a longer focal length lens position. Thus many camera exposure programs select shorter exposure times for these long focal length conditions and balance the exposure by increasing EI, using a larger aperture or both. Similar blurring effects occur with excessive subject motion. Many camera systems include an image stabilization system which can significantly reduce but not eliminate the effects of camera motion, however such systems can do nothing about subject motion.
At lower light levels, tradeoffs are typically required to obtain a properly exposed image. Increasing the light by adjusting the aperture setting is generally not an option as the aperture size is nominally set to its maximum by default (i.e., the aperture setting is set at the minimum F/#). Increasing the exposure time setting increases susceptibility to blur, but increasing the EI setting introduces higher levels of image noise. The aforementioned exposure program optimization generally makes some assumptions about camera motion and subject motion, taking into account the effects of zoom and image stabilization, to find the optimal exposure time and ISO tradeoff.
More sophisticated camera systems may compute an estimate of subject motion in order to determine the optimal exposure time. Various methods of estimating motion are available to those skilled in the art, the most common of which is to capture two images separated in time and measure the change in spatial location of objects between frames. One such method is described by De Haan in U.S. Pat. No. 5,929,919, entitled “Motion-compensated field rate conversion.” Such methods are heavily exploited in video image processing to reduce the jittering which results from rapid movement of the image from frame to frame. In video systems the average global shift of the frame is generally sufficient for this purpose. For the purpose of exposure program control, a more sophisticated analysis of the various motions within a scene is desirable in order to determine the main subject of interest and thereby determine the ideal exposure time for that object.
U.S. Pat. No. 7,657,164 to Nomura et al., entitled “Subject shake detection device, imaging device, control method thereof, control program, and recording medium,” describe the use of gyros and image analysis to estimate camera shake. The exposure time is adjusted to limit motion blur according to a predefined threshold.
U.S. Pat. No. 7,720,376 to Weinberg et al., entitled “Camera with acceleration sensor,” teaches a camera with an acceleration sensor. A sensed acceleration is used in the process of determining a minimum shutter speed that should be used for a particular focal length.
U.S. Patent Application Publication 2007/0188617 to Stavely, entitled “Apparatus and method for reducing image blur in a digital camera,” teaches determining camera motion information using motion sensors and image analysis. The motion information is used to control the moment of image capture to provide reduced motion blur.
U.S. Patent Application Publication 2007/0237514 to Pillman et al., entitled “Varying camera self-determination based on subject motion,” teaches a method for capturing digital images where motion in the scene is measured prior to image capture. Various camera settings are adjusted responsive to the determined scene motion. If little or no scene motion is detected, additional analysis is done to help select a capture mode setting for the digital camera. If rapid scene motion is detected, a capture mode setting suitable for sports photography is selected by the exposure control system. The sports capture mode limits the exposure time and uses a higher exposure index setting than a typical default capture mode. As such, the method of Pullman primarily provides an improved method for capture of scenes with significant scene motion.
In U.S. Patent Application Publication 2007/0237506 to Minema et al., entitled “Image blurring reduction,” a camera is described wherein an image is captured at a slower shutter speed if no camera motion is detected. If camera motion is detected, then an image is captured at a faster shutter speed. While this method does reduce motion blur in images, it does not address the combined effects of motion blur and noise in the image on the perceived image quality of the image in selecting capture conditions including exposure time and ISO.
U.S. Pat. No. 5,598,237 to McIntyre et al., entitled “Image capture apparatus,” describes an image capture apparatus operable in a hand-held condition and in a stabilized non-hand-held condition. Different exposure parameters are selected depending on whether the camera is being used in the hand-held condition.
U.S. Patent Application Publication 2009/0040364 to Rubner, entitled “Adaptive Exposure Control,” teaches using a multiple image capture process to reduce image quality artifacts including motion blur. In capture first image step, a first image is captured using exposure conditions defined by a camera auto exposure control system. In an analyze image for deficiencies step, the first image is analyzed for aspects of image quality such as overexposure, underexposure, motion blur, dynamic range or depth of field to determine which aspects have been met and where deficiencies remain. Based on this analysis, a remaining deficiencies test is used to check whether any deficiencies remain in the aspects of image quality. If some deficiencies remain, the process proceeds to update exposure parameters step where new exposure parameters are set for at least one additional image. A capture additional image step is then used to capture an additional image using the new exposure parameters. The additional image is then analyzed with the analyze image for deficiencies step. This process repeats until the remaining deficiencies test determines that all the aspects of image quality have been met amongst the multiple images that have been captured. A final image is then constructed by combining portions of the multiple captured images using a combine captured images step in such a way that all of the desired aspects of image quality are met. However, the method of Rubner does not address motion related image quality issues in applications which require capturing only a single captured image.
U.S. Patent Application Publication 2007/0236567 to Pillman et al., entitled “Camera and method with additional evaluation image capture based on scene brightness changes,” teaches capturing a set of evaluation images of a scene with a digital camera. A change in scene brightness between the evaluation images is determined and when the brightness change is outside a predetermined range; the camera is set to a different capture state prior capturing additional evaluation images of said set. A final camera state for capturing a final image is determined using said set of evaluation images.
U.S. Patent Application Publication 2009/0244301 to Border et al., entitled “Controlling multiple-image capture,” teaches determining pre-capture information including motion estimates by analyzing preview images. The method determines whether multiple image capture is appropriate based upon an analysis of the pre-capture information. The multiple images are then synthesized into a single image.
U.S. Pat. No. 7,546,026 to Pertsel, et al., entitled “Camera exposure optimization techniques that take camera motion and scene motion into account,” describes exposure optimization using both camera motion and scene motion information. This approach adjusts exposure settings using both global (camera motion) and local (scene motion) estimates. The use of fixed locations for local motion estimates can produce spurious motion estimates in regions of the image with insufficient detail to provide meaningful motion estimates. Additionally, the system is limited in its ability to respond automatically to scene content; in particular, it has no ability to account for differences in the location of the main subject and the amount of scene detail.
There remains a need for a method to adjust image capture settings for an electronic image capture device to provide improved image quality for images containing moving objects.